Control nucleic acid sequences for use in sequencing-by-synthesis and methods for designing the same

ABSTRACT

A method for nucleic acid sequencing includes (a) disposing a plurality of template polynucleotide strands in a plurality of defined spaces disposed on a sensor array, at least some of the template polynucleotide strands comprising a test or control sequence; (b) exposing a plurality of the template polynucleotide strands in the defined spaces to a series of flows of nucleotide species flowed according to a predetermined ordering; and (c) determining sequence information for a plurality of the template polynucleotide strands in the defined spaces based on the flows of nucleotide species to generate a plurality of sequencing reads corresponding to the template polynucleotide strands, wherein the test or control sequence comprises a sequence determined by identifying, using a variant caller, loci with systematic errors present in a plurality of sequencing runs included in a training set of sequencing runs.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Prov. Pat. Appl. No.61/858,828, filed Jul. 26, 2013, which is incorporated by referenceherein in its entirety.

FIELD

This application generally relates to control nucleic acid sequences andmethods for designing the same, and, more specifically, to controlnucleic acid sequences for use in sequencing-by-synthesis and methodsfor designing the same using a variant caller to identify loci withsystematic errors.

BACKGROUND

Control nucleic acid sequences may sometimes be used to facilitateassessment and/or analysis of nucleic acid sequencing data obtained invarious ways, including using next-generation sequencing systems suchas, for example, the Ion PGM™ and Ion Proton™ systems implementing IonTorrent™ sequencing technology (see, e.g., U.S. Pat. No. 7,948,015 andU.S. Pat. Appl. Publ. Nos. 2010/0137143, 2009/0026082, and 2010/0282617,which are all incorporated by reference herein in their entirety). Forexample, certain relatively short (e.g., less than 100 base pairs)nucleic acid sequences constrained to contain homopolymers of onlycertain lengths (e.g., homopolymers of length two, three, or four;homopolymers of length no more than 2; or homopolymers of length no morethan 1) may be used to attempt to assess potential error failure modesthat may be related to homopolymer of such lengths and may moregenerally be indicative of performance. However, these nucleic acidsequences may in some cases be oversensitive and may not be able toproperly capture or detect certain error modes of interest. There is aneed for new and improved control nucleic acid sequences and methods fordesigning the same that can better facilitate assessment and/or analysisof nucleic acid sequencing data obtained using the above-mentionedsystems or other sequencing systems/platforms.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and form a partof the specification, illustrate one or more exemplary embodiments andserve to explain the principles of various exemplary embodiments. Thedrawings are exemplary and explanatory only and are not to be construedas limiting or restrictive in any way.

FIG. 1 illustrates an exemplary system for nucleic acid sequencingand/or analysis.

FIG. 2 illustrates exemplary components of an apparatus for nucleic acidsequencing.

FIG. 3 illustrates an exemplary flow cell for nucleic acid sequencing.

FIG. 4 illustrates an exemplary process for label-free, pH-basedsequencing.

FIG. 5 illustrates an exemplary computer system.

FIG. 6 illustrates an exemplary method for nucleic acid sequencing usingcontrol sequences.

FIG. 7 illustrates an exemplary method for generating control sequences.

FIG. 8 illustrates an exemplary method for generating control sequences.

FIGS. 9A and 9B show plots of unique variants according to nucleotideand homopolymer length.

SUMMARY

According to an exemplary embodiment, there is provided a method fornucleic acid sequencing, comprising: (a) disposing a plurality oftemplate polynucleotide strands in a plurality of defined spacesdisposed on a sensor array, at least some of the template polynucleotidestrands comprising a test or control sequence; (b) exposing a pluralityof the template polynucleotide strands in the defined spaces to a seriesof flows of nucleotide species flowed according to a predeterminedordering; and (c) determining sequence information for a plurality ofthe template polynucleotide strands in the defined spaces based on theflows of nucleotide species to generate a plurality of sequencing readscorresponding to the template polynucleotide strands, wherein the testor control sequence comprises a sequence determined by identifying,using a variant caller, loci with systematic errors present in aplurality of sequencing runs included in a training set of sequencingruns.

According to an exemplary embodiment, there is provided a system,including: a plurality of template polynucleotide strands disposed in aplurality of defined spaces disposed on a sensor array, at least some ofthe template polynucleotide strands comprising a test or controlsequence, wherein the test or control sequence comprises a sequencedetermined by identifying, using a variant caller, loci with systematicerrors present in a plurality of sequencing runs included in a trainingset of sequencing runs; a machine-readable memory; and a processorconfigured to execute machine-readable instructions, which, whenexecuted by the processor, cause the system to perform a method fornucleic acid sequencing, comprising: (a) exposing a plurality of thetemplate polynucleotide strands in the defined spaces to a series offlows of nucleotide species flowed according to a predeterminedordering; and (b) determining sequence information for a plurality ofthe template polynucleotide strands in the defined spaces based on theflows of nucleotide species to generate a plurality of sequencing readscorresponding to the template polynucleotide strands.

According to an exemplary embodiment, there is provided a method fordesigning test or control sequences, comprising: identifying, using avariant caller, loci with systematic errors present in a plurality ofsequencing runs included in a training set of sequencing runs obtainedusing sequencing-by-synthesis; selecting a representative set of loci,including selecting from the identified loci an approximately equalnumber of loci involving errors in A, T, C, and G homopolymers andselecting from the identified loci an approximately equal number of lociinvolving homopolymers having a length of two, three, and four.

EXEMPLARY EMBODIMENTS

The following description and the various embodiments described hereinare exemplary and explanatory only and are not to be construed aslimiting or restrictive in any way. Other embodiments, features,objects, and advantages of the present teachings will be apparent fromthe description and accompanying drawings, and from the claims.

According to various exemplary embodiments, control nucleic acidsequences for test fragments and/or in line controls, and methods fordesigning the same, are disclosed herein. Such control nucleic acidsequences and methods for designing the same may improve the ability ofcontrol nucleic acid sequences to allow identification of compromisedsequencing experiments that produce data of substandard quality as aresult of sequencing failure modes. Such sequencing failure modes mayinclude sequencing failure modes that lead to reduced accuracy, whichmay include one or more sequencing failure modes such as: systematicerrors for high homopolymers in general, systematic errors for highhomopolymers in specific contexts, and/or systematic errors for specific“difficult” sequences not involving high homopolymers. Such controlnucleic acid sequences and methods for designing the same may helpdetect and/or reduce certain systematic errors and improve overallsequencing accuracy (especially in the case of long homopolymers), whichmay in turn improve downstream processing such as variant calling.

FIG. 1 illustrates an exemplary system for nucleic acid sequencingand/or analysis. The system includes an apparatus or sub-system fornucleic acid sequencing and/or analysis 11, a computingserver/node/device 12 including a base calling engine 13, arecalibration engine 14, a post-processing engine 15, and a display 16,which may be internal and/or external. The apparatus or sub-system fornucleic acid sequencing and/or analysis 11 may be any type of instrumentthat can generate nucleic acid sequence data from nucleic acid samples,which may include a nucleic acid sequencing instrument, areal-time/digital/quantitative PCR instrument, a microarray scanner,etc. The nucleic acid samples may include control/test nucleic acidsamples as further described herein and/or library nucleic acid samples.The computing server/node/device 12 may be a workstation, mainframecomputer, distributed computing node (part of a “cloud computing” ordistributed networking system), personal computer, mobile device, etc.The base calling engine 13 may be any suitable base caller and may beconfigured to include various signal/data processing modules that may beconfigured to receive signal/data from the apparatus or sub-system fornucleic acid sequencing and/or analysis 11 and perform variousprocessing steps, such as conversion from flow space to base space,determination of base calls for some or the entirety of a sequencingdata set, and determination of base call quality values. In anembodiment, the base calling engine 13 may implement one or morefeatures described in Davey et al., U.S. Pat. Appl. Publ. No.2012/0109598, published on May 3, 2012, and/or Sikora et al., U.S. Pat.Appl. Publ. No. 2013/0060482, published on Mar. 7, 2013, which are allincorporated by reference herein in their entirety. The base callingengine 13 may also include a mapping or alignment module for mapping oraligning reads to a reference sequence or genome, which may be awhole/partial genome, whole/partial exome, etc. In an embodiment, themapping or alignment module may include any suitable aligner, includingthe Torrent Mapping Alignment Program (TMAP), for example. Therecalibration engine 14 may be configured to recalibrate base calls orrelated intensity values or parameters based on an analysis of basecalling and alignment performed by the base calling engine 13, whichrecalibrated base calls or related intensity values or thresholds orparameters may be fed back into the base calling engine 13 for improvingthe accuracy of base calls. In an embodiment, the recalibration engine14 may implement one or more features described in Jiang et al., U.S.patent application Ser. No. 14/255,528, filed on Apr. 17, 2014, which isincorporated by reference herein in its entirety. The exemplary systemmay also include a client device terminal 17, which may include a dataanalysis API or module and may be communicatively connected to thecomputing server/node/device 12 via a network connection 18 that may bea “hardwired” physical network connection (e.g., Internet, LAN, WAN,VPN, etc.) or a wireless network connection (e.g., Wi-Fi, WLAN, etc.).The post-processing engine 15 may be configured to include varioussignal/data processing modules that may be configured to make variantcalls and apply post-processing to variant calls, which may includeannotating various variant calls and/or features, converting data fromflow space to base space, filtering of variants, and formatting thevariant data for display or use by client device terminal 17. Variantcalls may be made using any suitable variant caller, including theGerm-Line Variant Caller and the Torrent Variant Caller (TVC) Plug-insfor Ion Torrent™ sequencing technology. In an embodiment, the variantcaller may implement one or more features described in Hubbell et al.,U.S. patent application Ser. No. 14/200,942, filed Mar. 7, 2014, whichis incorporated by reference herein in its entirety. In an embodiment,the apparatus or sub-system for nucleic acid sequencing and/or analysis11 and the computing server/node/device 12 may be integrated into asingle instrument or system comprising components present in a singleenclosure 19. The client device terminal 17 may be configured tocommunicate information to and/or control the operation of the computingserver/node/device 12 and its modules and/or operating parameters.

FIG. 2 illustrates exemplary components of an apparatus for nucleic acidsequencing. Such an apparatus could be used as apparatus or sub-systemfor nucleic acid sequencing and/or analysis 11 of FIG. 1. The componentsinclude a flow cell and sensor array 100, a reference electrode 108, aplurality of reagents 114, a valve block 116, a wash solution 110, avalve 112, a fluidics controller 118, lines 120/122/126, passages104/109/111, a waste container 106, an array controller 124, and a userinterface 128. The flow cell and sensor array 100 includes an inlet 102,an outlet 103, a microwell array 107, and a flow chamber 105 defining aflow path of reagents over the microwell array 107. The referenceelectrode 108 may be of any suitable type or shape, including aconcentric cylinder with a fluid passage or a wire inserted into a lumenof passage 111. The reagents 114 may be driven through the fluidpathways, valves, and flow cell by pumps, gas pressure, or othersuitable methods, and may be discarded into the waste container 106after exiting the flow cell and sensor array 100. The reagents 114 may,for example, contain dNTPs to be flowed through passages 130 and throughthe valve block 116, which may control the flow of the reagents 114 toflow chamber 105 (also referred to herein as a reaction chamber) viapassage 109. The system may include a reservoir 110 for containing awash solution that may be used to wash away dNTPs, for example, that mayhave previously been flowed. The microwell array 107 may include anarray of defined spaces, such as microwells, for example, that isoperationally associated with a sensor array so that, for example, eachmicrowell has a sensor suitable for detecting an analyte or reactionproperty of interest. The defined spaces may include control/testnucleic acid samples as further described herein and/or library nucleicacid samples. The microwell array 107 may preferably be integrated withthe sensor array as a single device or chip. The array controller 124may provide bias voltages and timing and control signals to the sensor,and collect and/or process output signals. The user interface 128 maydisplay information from the flow cell and sensor array 100 as well asinstrument settings and controls, and allow a user to enter or setinstrument settings and controls. The valve 112 may be shut to preventany wash solution 110 from flowing into passage 109 as the reagents areflowing. Although the flow of wash solution may be stopped, there maystill be uninterrupted fluid and electrical communication between thereference electrode 108, passage 109, and the sensor array 107. Thefluidics controller 118 may be programmed to control driving forces forflowing reagents 114 and the operation of valve 112 and valve block 116to deliver reagents to the flow cell and sensor array 100 according to apredetermined reagent flow ordering.

In this application, “defined space” generally refers to any space(which may be in one, two, or three dimensions) in which at least someof a molecule, fluid, and/or solid can be confined, retained and/orlocalized. The space may be a predetermined area (which may be a flatarea) or volume, and may be defined, for example, by a depression or amicro-machined well in or associated with a microwell plate, microtiterplate, microplate, or a chip, or by isolated hydrophobic areas on agenerally hydrophobic surface. Defined spaces may be arranged as anarray, which may be a substantially planar one-dimensional ortwo-dimensional arrangement of elements such as sensors or wells.Defined spaces, whether arranged as an array or in some otherconfiguration, may be in electrical communication with at least onesensor to allow detection or measurement of one or more detectable ormeasurable parameter or characteristics. The sensors may convert changesin the presence, concentration, or amounts of reaction by-products (orchanges in ionic character of reactants) into an output signal, whichmay be registered electronically, for example, as a change in a voltagelevel or a current level which, in turn, may be processed to extractinformation or signal about a chemical reaction or desired associationevent, for example, a nucleotide incorporation event and/or a relatedion concentration (e.g., a pH measurement). The sensors may include atleast one ion sensitive field effect transistor (“ISFET”) or chemicallysensitive field effect transistor (“chemFET”).

FIG. 3 illustrates an exemplary flow cell for nucleic acid sequencing.The flow cell 200 includes a microwell array 202, a sensor array 205,and a flow chamber 206 in which a reagent flow 208 may move across asurface of the microwell array 202, over open ends of microwells in themicrowell array 202. The flow of reagents (e.g., nucleotide species) canbe provided in any suitable manner, including delivery by pipettes, orthrough tubes or passages connected to a flow chamber. A microwell 201in the microwell array 202 may have any suitable volume, shape, andaspect ratio. A sensor 214 in the sensor array 205 may be an ISFET or achemFET sensor with a floating gate 218 having a sensor plate 220separated from the microwell interior by a passivation layer 216, andmay be predominantly responsive to (and generate an output signalrelated to) an amount of charge 224 present on the passivation layer 216opposite of the sensor plate 220. Changes in the amount of charge 224cause changes in the current between a source 221 and a drain 222 of thesensor 214, which may be used directly to provide a current-based outputsignal or indirectly with additional circuitry to provide a voltageoutput signal. Reactants, wash solutions, and other reagents may moveinto microwells primarily by diffusion 240. One or more analyticalreactions to identify or determine characteristics or properties of ananalyte of interest may be carried out in one or more microwells of themicrowell array 202. Such reactions may generate directly or indirectlyby-products that affect the amount of charge 224 adjacent to the sensorplate 220. In an embodiment, a reference electrode 204 may be fluidlyconnected to the flow chamber 206 via a flow passage 203. In anembodiment, the microwell array 202 and the sensor array 205 maytogether form an integrated unit forming a bottom wall or floor of theflow cell 200. In an embodiment, one or more copies of an analyte may beattached to a solid phase support 212, which may include microparticles,nanoparticles, beads, gels, and may be solid and porous, for example.The analyte may include one or more copies of a nucleic acid analyte,which may include a control/test nucleic acid sample as furtherdescribed herein and/or a library nucleic acid sample, obtained usingany suitable technique.

FIG. 4 illustrates an exemplary process for label-free, pH-basedsequencing. A template 682 with sequence 685 and a primer binding site681 are attached to a solid phase support 680. The template 682 maycomprise a control/test nucleic acid sample as further described hereinand/or a library nucleic acid sample. The template 682 may be attachedas a clonal population to a solid support, such as a microparticle orbead, for example, and may be prepared as disclosed in Leamon et al.,U.S. Pat. No. 7,323,305. In an embodiment, the template may beassociated with a substrate surface or present in a liquid phase with orwithout being coupled to a support. A primer 684 and DNA polymerase 686are annealed to the template 682 so that the primer's 3′ end may beextended by a polymerase and that a polymerase is bound to suchprimer-template duplex (or in close proximity thereof) so that bindingand/or extension may take place when dNTPs are added. In step 688, dNTP(shown as dATP) is added, and the DNA polymerase 686 incorporates anucleotide “A” (since “T” is the next nucleotide in the template 682 andis complementary to the flowed dATP nucleotide). In step 690, a wash isperformed. In step 692, the next dNTP (shown as dCTP) is added, and theDNA polymerase 686 incorporates a nucleotide “C” (since “G” is the nextnucleotide in the template 682). More details about pH-based nucleicacid sequencing may be found in U.S. Pat. No. 7,948,015 and U.S. Pat.Appl. Publ. Nos. 2010/0137143, 2009/0026082, and 2010/0282617.

In an embodiment, the primer-template-polymerase complex may besubjected to a series of exposures of different nucleotides in apre-determined sequence or ordering. If one or more nucleotides areincorporated, then the signal resulting from the incorporation reactionmay be detected, and after repeated cycles of nucleotide addition,primer extension, and signal acquisition, the nucleotide sequence of thetemplate strand may be determined. The output signals measuredthroughout this process depend on the number of nucleotideincorporations. Specifically, in each addition step, the polymeraseextends the primer by incorporating added dNTP only if the next base inthe template is complementary to the added dNTP. With eachincorporation, an hydrogen ion is released, and collectively apopulation released hydrogen ions change the local pH of the reactionchamber. The production of hydrogen ions may be monotonically related tothe number of contiguous complementary bases (e.g., homopolymers) in thetemplate. Deliveries of nucleotides to a reaction vessel or chamber maybe referred to as “flows” of nucleotide triphosphates (or dNTPs). Forconvenience, a flow of dATP will sometimes be referred to as “a flow ofA” or “an A flow,” and a sequence of flows may be represented as asequence of letters, such as “ATGT” indicating “a flow of dATP, followedby a flow of dTTP, followed by a flow of dGTP, followed by a flow ofdTTP.” The predetermined ordering may be based on a cyclical, repeatingpattern consisting of consecutive repeats of a short pre-determinedreagent flow ordering (e.g., consecutive repeats of pre-determinedsequence of four nucleotide reagents such as, for example, “ACTG ACTG .. . ”), may be based in whole or in part on some other pattern ofreagent flows (such as, e.g., any of the various reagent flow orderingsdiscussed in Hubbell et al., U.S. Pat. Appl. Publ. No. 2012/0264621,published Oct. 18, 2012, which is incorporated by reference herein inits entirety), and may also be based on some combination thereof.

In various embodiments, output signals due to nucleotide incorporationmay be processed, given knowledge of what nucleotide species were flowedand in what order to obtain such signals, to make base calls for theflows and compile consecutive base calls associated with a samplenucleic acid template into a read. A base call refers to a particularnucleotide identification (e.g., dATP (“A”), dCTP (“C”), dGTP (“G”), ordTTP (“T”)). Base calling may include performing one or more signalnormalizations, signal phase and signal decay (e.g, enzyme efficiencyloss) estimations, signal corrections, and model-based signalpredictions, and may identify or estimate base calls for each flow foreach defined space. Any suitable base calling method may be used,including as described in Davey et al., U.S. Pat. Appl. Publ. No.2012/0109598, published on May 3, 2012, and/or Sikora et al., U.S. Pat.Appl. Publ. No. 2013/0060482, published on Mar. 7, 2013, which are allincorporated by reference herein in their entirety, recognizing ofcourse that more accurate base callers may yield better results.

FIG. 5 illustrates an exemplary computer system. Such a computer systemcould be used as computing server/node/device 12 of FIG. 1. The computersystem 501 includes a bus 502 or other communication mechanism forcommunicating information, a processor 503 coupled to the bus 502 forprocessing information, and a memory 505 coupled to the bus 502 fordynamically and/or statically storing information. The computer system501 can also include one or more co-processors 504 coupled to the bus502, such as GPUs and/or FPGAs, for performing specialized processingtasks; a display 506 coupled to the bus 502, such as a cathode ray tube(CRT) or liquid crystal display (LCD), for displaying information to acomputer user; an input device 507 coupled to the bus 502, such as akeyboard including alphanumeric and other keys, for communicatinginformation and command selections to the processor 503; a cursorcontrol device 508 coupled to the bus 502, such as a mouse, a trackballor cursor direction keys for communicating direction information andcommand selections to the processor 503 and for controlling cursormovement on display 506; and one or more storage devices 509 coupled tothe bus 502, such as a magnetic disk or an optical disk, for storinginformation and instructions. The memory 505 may include a random accessmemory (RAM) or other dynamic storage device and/or a read only memory(ROM) or other static storage device. Such an exemplary computer systemwith suitable software may be used to perform the embodiments describedherein. More generally, in various embodiments, one or more features ofthe teachings and/or embodiments described herein may be performed orimplemented using appropriately configured and/or programmed hardwareand/or software elements.

Examples of hardware elements may include processors, microprocessors,input(s) and/or output(s) (I/O) device(s) (or peripherals) that arecommunicatively coupled via a local interface circuit, circuit elements(e.g., transistors, resistors, capacitors, inductors, and so forth),integrated circuits, application specific integrated circuits (ASIC),programmable logic devices (PLD), digital signal processors (DSP), fieldprogrammable gate array (FPGA), logic gates, registers, semiconductordevice, chips, microchips, chip sets, and so forth. The local interfacemay include, for example, one or more buses or other wired or wirelessconnections, controllers, buffers (caches), drivers, repeaters andreceivers, etc., to allow appropriate communications between hardwarecomponents. A processor is a hardware device for executing software,particularly software stored in memory. The processor can be any custommade or commercially available processor, a central processing unit(CPU), an auxiliary processor among several processors associated withthe computer, a semiconductor based microprocessor (e.g., in the form ofa microchip or chip set), a macroprocessor, or generally any device forexecuting software instructions. A processor can also represent adistributed processing architecture. The I/O devices can include inputdevices, for example, a keyboard, a mouse, a scanner, a microphone, atouch screen, an interface for various medical devices and/or laboratoryinstruments, a bar code reader, a stylus, a laser reader, aradio-frequency device reader, etc. Furthermore, the I/O devices alsocan include output devices, for example, a printer, a bar code printer,a display, etc. Finally, the I/O devices further can include devicesthat communicate as both inputs and outputs, for example, amodulator/demodulator (modem; for accessing another device, system, ornetwork), a radio frequency (RF) or other transceiver, a telephonicinterface, a bridge, a router, etc.

Examples of software may include software components, programs,applications, computer programs, application programs, system programs,machine programs, operating system software, middleware, firmware,software modules, routines, subroutines, functions, methods, procedures,software interfaces, application program interfaces (API), instructionsets, computing code, computer code, code segments, computer codesegments, words, values, symbols, or any combination thereof. A softwarein memory may include one or more separate programs, which may includeordered listings of executable instructions for implementing logicalfunctions. The software in memory may include a system for identifyingdata streams in accordance with the present teachings and any suitablecustom made or commercially available operating system (O/S), which maycontrol the execution of other computer programs such as the system, andprovides scheduling, input-output control, file and data management,memory management, communication control, etc.

According to various embodiments, one or more features of teachingsand/or embodiments described herein may be performed or implementedusing an appropriately configured and/or programmed non-transitorymachine-readable medium or article that may store an instruction or aset of instructions that, if executed by a machine, may cause themachine to perform a method and/or operations in accordance with theembodiments. Such a machine may include, for example, any suitableprocessing platform, computing platform, computing device, processingdevice, computing system, processing system, computer, processor,scientific or laboratory instrument, etc., and may be implemented usingany suitable combination of hardware and/or software. Themachine-readable medium or article may include, for example, anysuitable type of memory unit, memory device, memory article, memorymedium, storage device, storage article, storage medium and/or storageunit, for example, memory, removable or non-removable media, erasable ornon-erasable media, writeable or re-writeable media, digital or analogmedia, hard disk, floppy disk, read-only memory compact disc (CD-ROM),recordable compact disc (CD-R), rewriteable compact disc (CD-RW),optical disk, magnetic media, magneto-optical media, removable memorycards or disks, various types of Digital Versatile Disc (DVD), a tape, acassette, etc., including any medium suitable for use in a computer.Memory can include any one or a combination of volatile memory elements(e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.)) andnonvolatile memory elements (e.g., ROM, EPROM, EEROM, Flash memory, harddrive, tape, CDROM, etc.). Moreover, memory can incorporate electronic,magnetic, optical, and/or other types of storage media. Memory can havea distributed, clustered, remote, or cloud architecture where variouscomponents are situated remote from one another, but are still accessedby the processor. The instructions may include any suitable type ofcode, such as source code, compiled code, interpreted code, executablecode, static code, dynamic code, encrypted code, etc., implemented usingany suitable high-level, low-level, object-oriented, visual, compiledand/or interpreted programming language.

FIG. 6 illustrates an exemplary method for nucleic acid sequencing usingcontrol sequences. In step 601, a user or component disposes a pluralityof template polynucleotide strands in a plurality of defined spacesdisposed on a sensor array, at least some of the template polynucleotidestrands comprising a test or control sequence comprising a sequencedetermined by identifying, using a variant caller, loci with systematicerrors present in a plurality of sequencing runs included in a trainingset of sequencing runs. Any suitable variant caller may be used,including the Germ-Line Variant Caller and the Torrent Variant Caller(TVC) Plug-ins for Ion Torrent™ sequencing technology, for example. Inan embodiment, the variant caller may implement one or more featuresdescribed in Hubbell et al., U.S. patent application Ser. No.14/200,942, filed Mar. 7, 2014, which is incorporated by referenceherein in its entirety. In step 602, a user or component exposes aplurality of the template polynucleotide strands in the defined spacesto a series of flows of nucleotide species flowed according to apredetermined ordering. Any suitable predetermined ordering may be used.For example, the predetermined ordering may be based on a cyclical,repeating pattern consisting of consecutive repeats of a shortpre-determined reagent flow ordering (e.g., consecutive repeats ofpre-determined sequence of four nucleotide reagents such as, forexample, “ACTG ACTG . . . ”), may be based in whole or in part on someother pattern of reagent flows (such as, e.g., any of the variousreagent flow orderings discussed in Hubbell et al., U.S. Pat. Appl.Publ. No. 2012/0264621, published Oct. 18, 2012, which is incorporatedby reference herein in its entirety), and may also be based on somecombination thereof. In step 603, a server or other computing means orresource determines sequence information for a plurality of the templatepolynucleotide strands in the defined spaces based on the flows ofnucleotide species to generate a plurality of sequencing readscorresponding to the template polynucleotide strands. The sequenceinformation may be determined using measured intensity values that maybe related to voltage data indicative of hydrogen ion concentrationsrepresentative of a number of nucleotide incorporations responsive toeach flowed nucleotide species or may include any other type of data(e.g., pyrophosphate, light, etc.) that could be representative of anumber of nucleotide incorporations responsive to each flowed nucleotidespecies. Any suitable base calling method may be used, including asdescribed in Davey et al., U.S. Pat. Appl. Publ. No. 2012/0109598,published on May 3, 2012, and/or Sikora et al., U.S. Pat. Appl. Publ.No. 2013/0060482, published on Mar. 7, 2013, which are all incorporatedby reference herein in their entirety.

Design of Control Sequences

In various embodiments, control nucleic acid sequences (e.g., for use insequencing-by-synthesis) may be designed based at least in part onlength considerations. For example, control nucleic acid sequences maybe designed to have a length in excess of 100 bases, such as at least125 bases, at least 150 bases, at least 175 bases, at least 200 bases,at least 225 bases, at least 250 bases, or more. Such control nucleicacid sequences may be less oversensitive to errors compared to libraryreads than would be shorter control nucleic acid sequences (e.g.,“short” sequences of 96 bases only allow one error when using a 50Q17quality metric), and may therefore provide a more quantitativeindication of run performance than shorter sequences.

In various embodiments, control nucleic acid sequences (e.g., for use insequencing-by-synthesis) may be designed with content other than arelatively short series of homopolymers of only certain lengths. Forexample, control nucleic acid sequences may be designed with contentthat is not merely a series of homopolymers of length two, three, orfour and no other length; or of homopolymers of length no more than 2;or of homopolymers of length no more than 1. Such control nucleic acidsequences with more complex content may be less sensitive to certainerrors specific to particular homopolymer lengths or other phenomenasuch as pH drift, and may be better adapted to assess actual sequencingerrors. In particular, such control nucleic acid sequences with morecomplex content may help provide improved determinations of generalpass/fail criteria, may help support longer inserts and have similarperformance/read length, and may provide sequences with single startingpoints that represent library read quality and that could start atmultiple points of a given sequence.

In various embodiments, control nucleic acid sequences (e.g., for use insequencing-by-synthesis) may be designed by identifying, using a variantcaller, loci with systematic errors present in a plurality of sequencingruns included in a training set of sequencing runs. The control nucleicacid sequences may be generated by performing a set of sequencing runsfor templates of a known library (e.g., E. coli DH10B, Lambda, syntheticsequences, etc.) under various conditions and analyzing an extent towhich each of the runs is affected by one or more sequencing failuremodes such as: systematic errors for high homopolymers in general,systematic errors for high homopolymers in specific contexts, and/orsystematic errors for specific “difficult” sequences not involving highhomopolymers. In an embodiment, control nucleic acid sequences may begenerated using combinations of sequence fragments that contain multipleinformative variants, such as context sequences that contain falsepositive variants (which may identified using variant calls made by avariant caller, as such calls would be false positives since the exactreference sequence is known). The particular combinations may be of adesired length, preferably above 100 bases. For example, thecombinations may be 125, 150, 175, 200, 225, 250, or more bases inlength (without adapters) comprising some segments that are known to bedifficult to sequence for some known library. The combinations may begenerated from identified segments in any suitable manner, includingrandomly in whole or in part. The size of the regions may vary and thelocation of the variant in the region may be at or near the center butthat is not necessary. Once the regions have been selected, they may bestitched together to form a desired number of sequences of some desiredlength (e.g., sets of 10 regions of 20-base fragments could be stitchedtogether to form 200-base control sequences).

FIG. 7 illustrates an exemplary method for generating control sequences.In step 701, a server or other computing means or resource identifies,using a variant caller, loci with systematic errors present in aplurality of sequencing runs included in a training set of sequencingruns obtained using sequencing-by-synthesis. In step 702, the server orother computing means or resource selects a representative set of loci,including selecting from the identified loci an approximately equalnumber of loci involving errors in A, T, C, and G homopolymers andselecting from the identified loci an approximately equal number of lociinvolving homopolymers having a length of two, three, and four.

FIG. 8 illustrates an exemplary method for generating control sequences.In step 801, a user or component obtains a set of high-throughput runsfor a known reference library (e.g., E. coli DB10H). The run data mayinclude, e.g., measured intensity values that may be related to voltagedata indicative of hydrogen ion concentrations representative of anumber of nucleotide incorporations or may include any other type ofdata (e.g., pyrophosphate, light, etc.) that could otherwise berepresentative of a number of nucleotide incorporations. The data may beprocessed and analyzed to make base calls using any suitable basecalling method or approach, as discussed above. In step 802, a server orother computing means or resource identifies systematic error loci inthe runs using a variant caller. Any suitable variant caller may beused, as discussed above. In step 803, the server or other computingmeans or resource extracts co-occurring loci. For example, loci that arepresent in at least a minimal number of runs (e.g., at least 3 runs) maybe extracted. In some cases, that minimal threshold may be only two ormay be larger (e.g., at least 4 runs, at least 5 runs, or more), and thethreshold may be selected depending on the sample size. In step 804, theserver or other computing means or resource discards non-informativeloci. For example, loci that are present in more than a maximal numberof runs (e.g., more than 20 runs) may suggest the presence of sources oferror other than the ones contemplated in the present design process andmay thus be discarded. In some cases, that maximal threshold may besmaller or larger than 20, and the threshold may be selected dependingon the sample size. In step 805, the server or other computing means orresource selects a set of representative loci from the loci remainingafter steps 803 and 804. For example, the set of representative loci maybe selected to comprise an approximately equal number of loci involvingerrors in A, T, C, and G homopolymers and an approximately equal numberof loci involving homopolymers having a length of two, three, and four.In some cases, an approximately equal number of certain homopolymertypes and/or lengths may be an equal number of each type and/or length,however, depending on the data and variant calls that may not always bepossible. In some cases, an approximately equal number of certainhomopolymer types and/or lengths may correspond to situations where atleast two of the homopolymer types and/or lengths are equal. In somecases, an approximately equal number of certain homopolymer types and/orlengths may correspond to situations where at least three of thehomopolymer types and/or lengths are equal. In other cases, anapproximately equal number of certain homopolymer types and/or lengthsmay correspond to situations where none of each homopolymer type and/orlength is allowed to have a number further distant from the mean ormedian of the numbers of the homopolymer types and/or lengths than somepre-defined threshold. In some cases, different combinations ofhomopolymer types or lengths could be used (e.g., an approximately equalnumber of loci involving homopolymers having a length of two, four, andsix; or having a length of three, four, and five; etc.) In step 806, theserver or other computing means or resource extracts a context sequencefrom the known reference library for each of the representative loci,which may be positioned at or near the center of its correspondingcontext sequence although that is not necessary and other positionswithin the context sequence are also possible. In step 807, theextracted context sequences are combined into longer control sequences,which combination may be done in any suitable way, including by manualinspection/selection or randomly in whole or in part.

In an example, control nucleic acid sequences were generated using thefollowing steps. In step 1, a set of 94 high-throughput E. coli DH10Bruns were obtained using the Ion PGM™ system implementing Ion Torrent™sequencing technology. In step 2, each of the 94 runs was analyzed toidentify DH10B loci with systematic errors by running a variant calleron the sequencing data (as mentioned previously, since the DH10B samplehas no variants relative to the reference, every called variant is infact a false positive caused by systematic errors). Here, variant callswere obtained for the runs using the Germ-Line Variant Caller Plug-in,however, any suitable variant caller could be used also. Altogether, theruns contained 32,110 variant calls and 13,044 unique variants. FIG. 9Ashows a plot of unique variants according to nucleotide (A, C, G, or T)and homopolymer length (0-mer through 12-mer). Among the uniquevariants, 3,464 were called in at least 3 runs and 99 were called in atleast 20 runs. In step 3, each of the identified DH10B loci was analyzedto determine whether it is present in at least three of the runs (inother words, co-occurring variant locations were identified). In step 4,informative variant locations were pre-selected from the loci identifiedin step 3 by discarding loci with variant calls in more than 20 of theruns (if a given locus was called a variant on too many runs, that locusis likely to be systematically called differently from the reference forreasons other than the error modes of interest here). FIG. 9B shows aplot of a subset of unique variants according to nucleotide (A, C, G, orT) and homopolymer length (0-mer through 12-mer). Shown in FIG. 9B areunique variants present in at least 3 runs and at most 20 runs. In step5, a representative set of 100 loci was selected from the loci remainingafter steps 3 and 4 by selecting an approximately equal number of lociinvolving errors in A, T, C, and G homopolymers, and an approximatelyequal number of loci involving 2-mers, 3-mers, and 4-mers. Here, theloci included 25 loci for each of A, T, C, and G homopolymers, and 39loci for each of homopolymers of length 3 and 4 with 22 loci forhomopolymers of length 2. Two additional loci having consecutivehomopolymers of length 5 and 2 were additionally identified. In step 6,for each locus in the representative set of loci selected in step 5, a25-base context sequence containing that locus was extracted, with thelocus being at or near the center of the context sequence. Appendix Iincludes a list of 102 exemplary 25-base context sequence fragmentstogether with the position of the middle locus in the DH10B genome, thehomopolymer locus (see boldface/underline) for which the contextsequence was extracted, and the type and length of the homopolymer locusfor which the context sequence was extracted. In step 7, 100 of thecontext sequences extracted in step 6 were combined in silico intogroups of ten to form ten control sequences of 250 bases. Appendix IIincludes a list of 10 control sequences together with the homopolymerloci (see boldface/underline) for which each of the context sequences inthe control sequence was extracted, and the types and lengths of thehomopolymer locus for which each of the context sequences in the controlsequence was extracted. For example, the first control sequence is acombination of context sequences extracted for A homopolymers of lengths2, 3, and 4; G homopolymers of lengths 2, 3, and 4; C homopolymers oflengths 3 and 4; and T homopolymers of lengths 3 and 4. The controlsequences may of course include additional homopolymers other than theones used to extract the context sequences they comprise; only thehomopolymers used to extract the context sequences are shownboldface/underline and counted in the loci types/lengths column ofAppendix II.

In various examples, control sequences generated as described above maybe used without further selection or they may be further testedempirically in various ways to select a smaller subset of desiredcontrol sequences for use in particular applications or insequencing-by-synthesis generally. As part of such testing and/orsequencing, the control sequences may be synthesized and attached at oneend to a sequencing adapter that may include a sequencing keyidentifying the sequence as a control sequence (e.g., CCAT CTCA TCCCTGCG TGTC TCCG ACAT CG, SEQ ID NO: 113), and at the other end to anotheradapter sequence (e.g., ATCA CCGA CTGC CCAT AGAG AGGA AAGC GGAG GCGTAGTG G, SEQ ID NO: 114). Sequence synthesis and attachment may be doneusing any suitable method known in the art. A series of feasibility runsmay then be performed using any suitable sequencing technology, and asubset of desired control sequences may be selected based on an analysisof the runs. In some cases, some of the runs may be performed in idealsituations while others are intentionally performed under inadequatesituations (e.g., by intentionally using an inadequate pH level whenusing Ion Torrent™ sequencing technology), and comparison of thebehavior of the control sequences across ideal/inadequate situations maybe used to identify control sequences that better conform or are moreconsistent with the underlying experimental situation. Selection of aparticular subset of control sequences may be based on various accuracycriteria (e.g., mean read length, fraction of aligned reads, error(s) atparticular positions, or other quality metrics), or platform-specificparameters or phenomena (e.g., pH drift), or other error sources orerror-reducing goals or objectives, or some combination thereof.

According to an exemplary embodiment, there is provided a method fornucleic acid sequencing, comprising: (a) disposing a plurality oftemplate polynucleotide strands in a plurality of defined spacesdisposed on a sensor array, at least some of the template polynucleotidestrands comprising a test or control sequence; (b) exposing a pluralityof the template polynucleotide strands in the defined spaces to a seriesof flows of nucleotide species flowed according to a predeterminedordering; and (c) determining sequence information for a plurality ofthe template polynucleotide strands in the defined spaces based on theflows of nucleotide species to generate a plurality of sequencing readscorresponding to the template polynucleotide strands, wherein the testor control sequence comprises a sequence determined by identifying,using a variant caller, loci with systematic errors present in aplurality of sequencing runs included in a training set of sequencingruns.

In such a method, the test or control sequence may comprise a sequencefurther determined by finding co-occurring variant locations present inat least three sequencing runs included in a training set of sequencingruns. The test or control sequence may comprise a sequence furtherdetermined by pre-selecting informative variant locations. The test orcontrol sequence may comprise a sequence further determined bydiscarding co-occurring variant locations present in more than twentysequencing runs included in a training set of sequencing runs. The testor control sequence may comprise a sequence further determined byselecting a representative set of loci, including selecting from the setof identified loci an approximately equal number of loci involvingerrors in A, T, C, and G homopolymers. The test or control sequence mayfurther comprise a sequence further determined by selecting from the setof identified loci an approximately equal number of loci involvinghomopolymers having a length of two, three, and four. The test orcontrol sequence may comprise a sequence further determined byextracting a context sequence containing each locus in therepresentative set of loci. The test or control sequence may comprise asequence further determined by combining in silico the extracted contextsequences. The test or control sequence may comprise a sequence furtherdetermined by attaching one or more sequencing adapters to the combinedsequence. The test or control sequence may comprise a sequence furtherdetermined by finding co-occurring variant locations present in at leastthree and no more than twenty sequencing runs included in a training setof sequencing runs. The test or control sequence may comprise a sequencefurther determined by finding co-occurring variant locations present inat least five and no more than fifteen sequencing runs included in atraining set of sequencing runs.

According to an exemplary embodiment, there is provided a kit fornucleic acid sequencing, comprising: a plurality of test or controlsequences each comprising a sequence determined by identifying, using avariant caller, loci with systematic errors present in a plurality ofsequencing runs included in a training set of sequencing runs obtainedusing sequencing-by-synthesis, wherein the test or control sequenceseach comprise a sequence further determined by selecting arepresentative set of loci, including selecting from the identified locian approximately equal number of loci involving errors in A, T, C, and Ghomopolymers and selecting from the identified loci an approximatelyequal number of loci involving homopolymers having a length of two,three, and four.

According to an exemplary embodiment, there is provided a system,including: a plurality of template polynucleotide strands disposed in aplurality of defined spaces disposed on a sensor array, at least some ofthe template polynucleotide strands comprising a test or controlsequence, wherein the test or control sequence comprises a sequencedetermined by identifying, using a variant caller, loci with systematicerrors present in a plurality of sequencing runs included in a trainingset of sequencing runs; a machine-readable memory; and a processorconfigured to execute machine-readable instructions, which, whenexecuted by the processor, cause the system to perform a method fornucleic acid sequencing, comprising: (a) exposing a plurality of thetemplate polynucleotide strands in the defined spaces to a series offlows of nucleotide species flowed according to a predeterminedordering; and (b) determining sequence information for a plurality ofthe template polynucleotide strands in the defined spaces based on theflows of nucleotide species to generate a plurality of sequencing readscorresponding to the template polynucleotide strands.

In such a system, the test or control sequence may comprise a sequencefurther determined by selecting a representative set of loci, includingselecting from the identified loci an approximately equal number of lociinvolving errors in A, T, C, and G homopolymers and selecting from theidentified loci an approximately equal number of loci involvinghomopolymers having a length of two, three, and four.

According to an exemplary embodiment, there is provided a method fordesigning test or control sequences, comprising: identifying, using avariant caller, loci with systematic errors present in a plurality ofsequencing runs included in a training set of sequencing runs obtainedusing sequencing-by-synthesis; and selecting a representative set ofloci, including selecting from the identified loci an approximatelyequal number of loci involving errors in A, T, C, and G homopolymers andselecting from the identified loci an approximately equal number of lociinvolving homopolymers having a length of two, three, and four.

In such a method, the test or control sequence may comprise a sequencefurther determined by finding co-occurring variant locations present inat least three sequencing runs included in a training set of sequencingruns. The test or control sequence may comprise a sequence furtherdetermined by pre-selecting informative variant locations. The test orcontrol sequence may comprise a sequence further determined bydiscarding co-occurring variant locations present in more than twentysequencing runs included in a training set of sequencing runs. The testor control sequence may comprise a sequence further determined byextracting a context sequence containing each locus in therepresentative set of loci. The test or control sequence may comprise asequence further determined by combining in silico the extracted contextsequences. The test or control sequence may comprise a sequence furtherdetermined by attaching one or more sequencing adapters to the combinedsequence. The test or control sequence may comprise a sequence furtherdetermined by finding co-occurring variant locations present in at leastthree and no more than twenty sequencing runs included in a training setof sequencing runs. The test or control sequence may comprise a sequencefurther determined by finding co-occurring variant locations present inat least five and no more than fifteen sequencing runs included in atraining set of sequencing runs.

Unless otherwise specifically designated herein, terms, techniques, andsymbols of biochemistry, cell biology, genetics, molecular biology,nucleic acid chemistry, nucleic acid sequencing, and organic chemistryused herein follow those of standard treatises and texts in the relevantfield.

Although the present description described in detail certainembodiments, other embodiments are also possible and within the scope ofthe present invention. For example, those skilled in the art mayappreciate from the present description that the present teachings maybe implemented in a variety of forms, and that the various embodimentsmay be implemented alone or in combination. Variations and modificationswill be apparent to those skilled in the art from consideration of thespecification and figures and practice of the teachings described in thespecification and figures, and the claims.

APPENDIX I Position Context Sequence Fragment Type Length SEQ ID NO:3059197 GATGCAGCACCG AA GGCTGAATATC A 2 SEQ ID NO: 1 3620936TCTCAGGTTACG AA GGCGGTGCCAA A 2 SEQ ID NO: 2 1321265 TTGATCGACTTT AACGTCCGTGCGG A 2 SEQ ID NO: 3 454206 AGCCCCGGCTGT AA CGTTTTGGTAT A 2SEQ ID NO: 4 1505557 GTTTGCCGAGGC AA TATATGTCCGG A 2 SEQ ID NO: 53930414 CCGCCGAAGGCC AA CCCCAGTTTGA A 2 SEQ ID NO: 6 3665932 ACAGCGGCGGGAAA TTTCCCACCTG A 3 SEQ ID NO: 7 4028930 GTGGGTCAGCG AAA CGTTTCGCTGA A 3SEQ ID NO: 8 3032774 AGCGGAACAGT AAA TTTACGGCAGA A 3 SEQ ID NO: 92183647 GTTATGAACCC AAA GTCAGCCGTGA A 3 SEQ ID NO: 10 2568253GACTGCCCTTT A AA CCTGTACCCAC A 3 SEQ ID NO: 11 2903059 TACTGGCAAAT AAAGTACGTTCCAC A 3 SEQ ID NO: 12 992689 AAGAGCGTCGT AAA GTATTGCAGGT A 3SEQ ID NO: 13 945598 TCAGGCGGCGG AAA GCGTGATTGAC A 3 SEQ ID NO: 141204000 CTGACGCTGCC AAA CGCCGACCGCG A 3 SEQ ID NO: 15 4284712ATGCGCGCGTT AAA GTGCGTATCAC A 3 SEQ ID NO: 16 1221988 ATGGTTACTTT AAAACCGGATTAAT A 4 SEQ ID NO: 17 2538868 CATCAGGCACC AAAA GAGTATGGCG A 4SEQ ID NO: 18 3186626 ACTTCGGCACC AAAA GCATTGGCGT A 4 SEQ ID NO: 194566931 GCGGGAAGGGG AAAA TCCATGCTGA A 4 SEQ ID NO: 20 4235391GTTCGTCCGTG AAAA TAAGAGTCAC A 4 SEQ ID NO: 21 887336 ATTTAAGTGAG AAAACCGGCAGCCA A 4 SEQ ID NO: 22 3480357 CGAAATTTGAT AAAA TCCCGCTCTT A 4SEQ ID NO: 23 2102606 GCTTGATCAGG AAAA GTTTGGTATC A 4 SEQ ID NO: 244286302 GTCCGGCACTG AAAA TCGTTGATGC A 4 SEQ ID NO: 25 1469524GCTGGCGGAGCC TT CAGTCTATTTT T 2 SEQ ID NO: 26 59306 CTGGAACGCCCC TTCAACCTTAGCA T 2 SEQ ID NO: 27 3395983 AAGGCGCAGGG TTT GCAGAGCTGTT T 3SEQ ID NO: 28 2781684 GCCACCAGCCC TTT GCTTTCCAGTG T 3 SEQ ID NO: 294533299 GTACCGGCAAA TTT GCCGCCGTAAG T 3 SEQ ID NO: 30 1817146GTCGGCGCAAA TTT GCAACCAGAAG T 3 SEQ ID NO: 31 160766 GCGTGACCAAA TTTGGTGCAGCGCC T 3 SEQ ID NO: 32 3665935 GCGGCGGGAAA TTT CCCACCTGATA T 3SEQ ID NO: 33 197121 CTCGCGGTAAA TTT ACCGAAGCACA T 3 SEQ ID NO: 341554427 TCCCAAACCGG TTT CGTTTAATAAT T 3 SEQ ID NO: 35 2039554TGGCGGCGAAA TTT CGCGCCAGCGG T 3 SEQ ID NO: 36 4180498 TCCGTTACACC TTTTCCACATTCAC T 4 SEQ ID NO: 37 881464 TGTGTCAGGGC TTTT GGTTCTCCCT T 4SEQ ID NO: 38 2659290 TCTGCAATTCA TTTT GCATATAGCC T 4 SEQ ID NO: 394099761 CATAACTATTG TTTT GATGAATCAG T 4 SEQ ID NO: 40 4497018GTATCGCCAGC TTTT GCAAACGCCC T 4 SEQ ID NO: 41 2308230 ACGCTGCATCG TTTTCATCTTTAAA T 4 SEQ ID NO: 42 2425780 AGATAGCTCCC TTTT GGCATGAAGA T 4SEQ ID NO: 43 4073210 TGTTGAACTAC TTTT CCTGATATGT T 4 SEQ ID NO: 443385917 AACCAGCACTC TTTT CATGGCTATC T 4 SEQ ID NO: 45 955284 ACAGGACGCCATTTT GCCGACTCCC T 4 SEQ ID NO: 46 1018410 TCTGGCGGCAA TTTT GCTGATGGAT T4 SEQ ID NO: 47 2212298 TCAATGGTGAC TTTT GCCGTTCCCG T 4 SEQ ID NO: 482380894 CTGCTGCCAGA TTTT CACCTGCTGA T 4 SEQ ID NO: 49 1765974CTTTGACACCA TTTT CCGTAGTGAA T 4 SEQ ID NO: 50 4596308 GTTCGAGTCCGG CCTTCGGCACCAA C 2 SEQ ID NO: 51 1638520 AGGGATGGGACG CC TGTTTGCCATC C 2SEQ ID NO: 52 821095 GATCGATCCAGG CC TAATCGATCGG C 2 SEQ ID NO: 533365511 ACGCTTATCAGG CC TACGCCATCTC C 2 SEQ ID NO: 54 777133GTGGTCAGCGAG CC ACGGGTCATCA C 2 SEQ ID NO: 55 2748576 AGCAGGTGACGG CCTTCATGATCGG C 2 SEQ ID NO: 56 3298858 TTGCGGCGGTAG CC AGCTGGAAGGA C 2SEQ ID NO: 57 4676206 CGATCGTCGCGG C C TGAATACCTGG C 2 SEQ ID NO: 582643681 GGAAAGCGATGG CC TACGGCGAGCG C 2 SEQ ID NO: 59 4466413TACCTGCGCCG CCC TGGTAGACGTC C 3 SEQ ID NO: 60 3778832 AGGCGACAATG CCCTGGTCTTTCGC C 3 SEQ ID NO: 61 52406 GGGCTAAGTGG CCC TGGTGGACTCG C 3SEQ ID NO: 62 792761 AGGCAATCGAG CCC AGATGCCGGAT C 3 SEQ ID NO: 634497766 CGTTGATTCTG CCC TTATTTACAAA C 3 SEQ ID NO: 64 1503858GAATAATCCAG CCC GCCAGGCATGG C 3 SEQ ID NO: 65 2926403 CAGGCAAGCCG CCCAGGTGCTCACA C 3 SEQ ID NO: 66 1857907 GTGTTTATCAT CCC TATTGCTTTGC C 3SEQ ID NO: 67 3638552 TATGGAAGCGG CCC AGATAAGCCAG C 3 SEQ ID NO: 682775787 TCAACGTGAAG CCC TGTTTAACGCT C 3 SEQ ID NO: 69 4167872TGATATTCCTG CCCC TGATAGCGGT C 4 SEQ ID NO: 70 2160039 ATGCCGCCAGT CCCCTGATGACCCG C 4 SEQ ID NO: 71 4275966 CGGTCGTGCGA CCCC GGTAGAGCTG C 4SEQ ID NO: 72 2878452 TGTTATATCTG CCCC GATAAAACGG C 4 SEQ ID NO: 733704512 AAGCCAATCAG CCCC TATCAACCGC C 4 SEQ ID NO: 74 2039072GTCACCTGCTG CCCC ACGTGGGACA C 4 SEQ ID NO: 75 3886677 CACAGGTGATAT GGCCTTCGCCAGA G 2 SEQ ID NO: 76 4257151 AGCTACCCGATA GG CTTCCGCCATC G 2SEQ ID NO: 77 2009720 TACGACTGCGAA GG CTTCTTCGTTG G 2 SEQ ID NO: 782018580 TGGGGCGGACAA GG CACTCGCGCCG G 2 SEQ ID NO: 79 3978844CAACGGGTTATA GG CACCGCCAGGG G 2 SEQ ID NO: 80 2937614 TAGCGGTAAAC GGGCTACCGGTATC G 3 SEQ ID NO: 81 1813728 TGATTGCAACA GGG CAAATTGCGCA G 3SEQ ID NO: 82 2716017 TGCATGAGGTC GGG TTGAATATCAA G 3 SEQ ID NO: 834190007 TTTCTGTTCCA GGG CTTCCGCCACC G 3 SEQ ID NO: 84 1217166ATGACGCCAGA GGG CTGGAGATGCA G 3 SEQ ID NO: 85 906184 TCGATCCTTGA GGGATGATTGCATT G 3 SEQ ID NO: 86 2247497 TGCGGCATACT GGG CTTCCGTATGC G 3SEQ ID NO: 87 8186 TGATATCATCA GGG CAGACCGGTTA G 3 SEQ ID NO: 88 2795077CACCAGAATCA GGG CAAACATATTC G 3 SEQ ID NO: 89 1807015 CCTGGTCTGGA GGGCAATACGCCCT G 3 SEQ ID NO: 90 1352099 ATCACCGAATC GGGG ACCACCGCCA G 4SEQ ID NO: 91 1048275 GCCCATAAATT GGGG CTGATCTCCA G 4 SEQ ID NO: 924685037 ACAGGCTAAGA GGGG CCGGACACCC G 4 SEQ ID NO: 93 968215 CCCTGAAGGCCGGGG CAGCCCACAT G 4 SEQ ID NO: 94 193277 GATTCGGCAAA GGGG AGATACGGTT G 4SEQ ID NO: 95 4561118 TATAGAGGATC GGGG CCACGCGCGC G 4 SEQ ID NO: 962141910 TCTTGCACAAA GGGG AGAAGCAATT G 4 SEQ ID NO: 97 4373401GAACGCTATCA GGGG CAAGTTTGCA G 4 SEQ ID NO: 98 4429478 CTTCCTCGATT GGGGCTGGCGTATT G 4 SEQ ID NO: 99 1925107 GGAATCGCCCT GGGG CGGCGCACAA G 4SEQ ID NO: 100 3147535 TATCCAAATTTTT GG CCGTTCACTG G 2 SEQ ID NO: 101364140 TTTGCTGGAAAAA TT GCGCGCCAAA T 2 SEQ ID NO: 102

APPENDIX II Control Sequence Loci Types (Lengths) SEQ ID NO:GATGCAGCACCG AA GGCTGAATATC A (2, 3, and 4) SEQ ID NO: 103 TAGCGGTAAACGGG CTACCGGTATC G (2, 3, and 4) TGATATTCCTG CCCC TGATAGCGGT C (3 and 4)GCCACCAGCCC TTT GCTTTCCAGTG T (3 and 4) CATCAGGCACC AAAA GAGTATGGCGTACCTGCGCCG CCC TGGTAGACGTC ACAGGCTAAGA GGGG CCGGACACCC GTTATGAACCC AAAGTCAGCCGTGA CATAACTATTG TTTT GATGAATCAG AGCTACCCGATA GG CTTCCGCCATCGCTGGCGGAGCC TT CAGTCTATTTT A (3 and 4) SEQ ID NO: 104 GGGCTAAGTGG CCCTGGTGGACTCG G (3 and 4) ATGGTTACTTT AAAA CCGGATTAAT C (2, 3, and 4)TGATTGCAACA GGG CAAATTGCGCA T (2, 3, and 4) TGTGTCAGGGC T TTT GGTTCTCCCTAGCGGAACAGT AAA TTTACGGCAGA CGGTCGTGCGA CCCC GGTAGAGCTG GTCGGCGCAAA TTTGCAACCAGAAG CCCTGAAGGCC GGGG CAGCCCACAT AGGGATGGGACG CC TGTTTGCCATCGTTCGAGTCCGG CC TTCGGCACCAA A (3 and 4) SEQ ID NO: 105 AAGGCGCAGGG TTTGCAGAGCTGTT G (3 and 4) ATCACCGAATC GGG G ACCACCGCCA C (2, 3, and 4)GTGGGTCAGCG AAA CGTTTCGCTGA T (2, 3, and 4) ATGCCGCCAGT CCCC TGATGACCCGTGCATGAGGTC GGG TTGAATATCAA TCTGCAATTCA TTTT GCATATAGCC AGGCAATCGAG CCCAGATGCCGGAT GCGGGAAGGGG AAAA TCCATGCTGA CTGGAACGCCCC TT CAACCTTAGCACACAGGTGATAT GG CCTTCGCCAGA A (2, 3, and 4) SEQ ID NO: 106 ACAGCGGCGGGAAA TTTCCCACCTG G (2, 3, and 4) TCCGTTACACC TTTT CCACATTCAC C (3 and 4)AGGCGACAATG CCC TGGTCTTTCGC T (3 and 4) GCCCATAAATT GGGG CTGATCTCCAGTACCGGCAAA TTT GCCGCCGTAAG ACTTCGGCACC AAAA GCATTGGCGT TTTCTGTTCCA GGGCTTCCGCCACC TGTTATATCTG CCCC GATAAAACGG TCTCAGGTTACG AA GGCGGTGCCAAGACTGCCCTTT AAA CCTGTACCCAC A (2, 3, and 4) SEQ ID NO: 107 TCGATCCTTGAGGG ATGATTGCATT G (3 and 4) CAGGCAAGCCG CCC AGGTGCTCACA C (2 and 3)TCCCAAACCGG TTT CGTTTAATAAT T (3, 4, and 4) GATTCGGCAAA GGGG AGATACGGTTACGCTGCATCG TTTT CATCTTTAAA CGAAATTTGAT AAAA TCCCGCTCTT AGCAGGTGACGG CCTTCATGATCGG TTGATCGACTTT AA CGTCCGTGCGG ACAGGACGCCA TTTT GCCGACTCCCGCGTGACCAAA TTT GGTGCAGCGCC A (2, 3, and 4) SEQ ID NO: 108 GAATAATCCAGCCC GCCAGGCATGG G (3 and 4) AAGAGCGTCGT AAA GTATTGCAGGT C (3 and 4)TGATATCATCA GGG CAGACCGGTTA T (3, 4, and 4) AAGCCAATCAG CCCC TATCAACCGCTATAGAGGATC GGGG CCACGCGCGC AGATAGCTCCC TTTT GGCATGAAGA GCTTGATCAGG AAAAGTTTGGTATC AACCAGCACTC TTTT CATGGCTATC AGCCCCGGCTGT AA CGTTTTGGTATCGTTGATTCTG CCC TTATTTACAAA A (3 and 4) SEQ ID NO: 109 GCGGCGGGAAA TTTCCCACCTGATA G (2, 3, and 4) TGCGGCATACT GGG CTTCCGTATGC C (2, 2, and 3)TCAGGCGGCGG AAA GCGTGATTGAC T (3 and 4) GTATCGCCAGC TTTT GCAAACGCCCATTTAAGTGAG AAAA CCGGCAGCCA GTGGTCAGCGAG CC ACGGGTCATCA GAACGCTATCA GGGGCAAGTTTGCA GATCGATCCAGG CC TAATCGATCGG TGGGGCGGACAA GG CACTCGCGCCGATGACGCCAGA GGG CTGGAGATGCA A (3 and 4) SEQ ID NO: 110 TACTGGCAAAT AAAGTACGTTCCAC G (2, 3, and 4) CTCGCGGTAAA TTT ACCGAAGCACA C (2, 3, and 4)GTGTTTATCAT CCC TATTGCTTTGC T (3 and 4) GTTCGTCCGTG AAAA TAAGAGTCACGTCACCTGCTG CCCC ACGTGGGACA TCTTGCACAAA GGGG AGAAGCAATT TGTTGAACTAC TTTTCCTGATATGT TACGACTGCGAA GG CTTCTTCGTTG ACGCTTATCAGG CC TACGCCATCTCGTCCGGCACTG AAAA TCGTTGATGC A (2 and 4) SEQ ID NO: 111 CTTTGACACCA TTTTCCGTAGTGAA G (2, 2, and 3) TTGCGGCGGTAG CC AGCTGGAAGGA C (2 and 3)CAACGGGTTATA GG CACCGCCAGGG T (4, 4, and 4) TATCCAAATTTTT GG CCGTTCACTGGTTTGCCGAGGC AA TATATGTCCGG TCTGGCGGCAA TTTT GCTGATGGAT TCAACGTGAAG CCCTGTTTAACGCT CACCAGAATCA GGG CAAACATATTC CTGCTGCCAGA TTTT CACCTGCTGATATGGAAGCGG CCC AGATAAGCCAG A (3 and 3) SEQ ID NO: 112 CCTGGTCTGGA GGGCAATACGCCCT G (3, 4, and 4) CTGACGCTGCC AAA CGCCGACCGCG C (2 and 3)TGGCGGCGAAA TTT CGCGCCAGCGG T (2, 3, and 4) TTTGCTGGAAAAA TT GCGCGCCAAACGATCGTCGCGG CC TGAATACCTGG CTTCCTCGATT GGGG CTGGCGTATT ATGCGCGCGTT AAAGTGCGTATCAC TCAATGGTGAC TTTT GCCGTTCCCG GGAATCGCCCT GGGG CGGCGCACAA

1. A method for nucleic acid sequencing, comprising: (a) disposing aplurality of template polynucleotide strands in a plurality of definedspaces disposed on a sensor array, at least some of the templatepolynucleotide strands comprising a test or control sequence; (b)exposing a plurality of the template polynucleotide strands in thedefined spaces to a series of flows of nucleotide species flowedaccording to a predetermined ordering; and (c) determining sequenceinformation for a plurality of the template polynucleotide strands inthe defined spaces based on the flows of nucleotide species to generatea plurality of sequencing reads corresponding to the templatepolynucleotide strands, wherein the test or control sequence comprises asequence determined by identifying, using a variant caller, loci withsystematic errors present in a plurality of sequencing runs included ina training set of sequencing runs.
 2. The method of claim 1, wherein thetest or control sequence comprises a sequence further determined byfinding co-occurring variant locations present in at least threesequencing runs included in a training set of sequencing runs.
 3. Themethod of claim 1, wherein the test or control sequence comprises asequence further determined by pre-selecting informative variantlocations.
 4. The method of claim 1, wherein the test or controlsequence comprises a sequence further determined by discardingco-occurring variant locations present in more than twenty sequencingruns included in a training set of sequencing runs.
 5. The method ofclaim 1, wherein the test or control sequence comprises a sequencefurther determined by selecting a representative set of loci, includingselecting from the identified loci an approximately equal number of lociinvolving errors in A, T, C, and G homopolymers.
 6. The method of claim5, wherein the test or control sequence comprises a sequence furtherdetermined by selecting from the identified loci an approximately equalnumber of loci involving homopolymers having a length of two, three, andfour.
 7. The method of claim 6, wherein the test or control sequencecomprises a sequence further determined by extracting a context sequencecontaining each locus in the representative set of loci.
 8. The methodof claim 7, wherein the test or control sequence comprises a sequencefurther determined by combining in silico the extracted contextsequences.
 9. The method of claim 8, wherein the test or controlsequence comprises a sequence further determined by attaching one ormore sequencing adapters to the combined sequence.
 10. A system,including: a plurality of template polynucleotide strands disposed in aplurality of defined spaces disposed on a sensor array, at least some ofthe template polynucleotide strands comprising a test or controlsequence, wherein the test or control sequence comprises a sequencedetermined by identifying, using a variant caller, loci with systematicerrors present in a plurality of sequencing runs included in a trainingset of sequencing runs; a machine-readable memory; and a processorconfigured to execute machine-readable instructions, which, whenexecuted by the processor, cause the system to perform a method fornucleic acid sequencing, comprising: (a) exposing a plurality of thetemplate polynucleotide strands in the defined spaces to a series offlows of nucleotide species flowed according to a predeterminedordering; and (b) determining sequence information for a plurality ofthe template polynucleotide strands in the defined spaces based on theflows of nucleotide species to generate a plurality of sequencing readscorresponding to the template polynucleotide strands.
 11. The system ofclaim 10, wherein the test or control sequence comprises a sequencefurther determined by selecting a representative set of loci, includingselecting from the identified loci an approximately equal number of lociinvolving errors in A, T, C, and G homopolymers and selecting from theidentified loci an approximately equal number of loci involvinghomopolymers having a length of two, three, and four.
 12. A method fordesigning test or control sequences, comprising: identifying, using avariant caller, loci with systematic errors present in a plurality ofsequencing runs included in a training set of sequencing runs obtainedusing sequencing-by-synthesis; and selecting a representative set ofloci, including selecting from the identified loci an approximatelyequal number of loci involving errors in A, T, C, and G homopolymers andselecting from the identified loci an approximately equal number of lociinvolving homopolymers having a length of two, three, and four.
 13. Themethod of claim 12, wherein the test or control sequence comprises asequence further determined by finding co-occurring variant locationspresent in at least three sequencing runs included in a training set ofsequencing runs.
 14. The method of claim 12, wherein the test or controlsequence comprises a sequence further determined by pre-selectinginformative variant locations.
 15. The method of claim 12, wherein thetest or control sequence comprises a sequence further determined bydiscarding co-occurring variant locations present in more than twentysequencing runs included in a training set of sequencing runs.
 16. Themethod of claim 12, wherein the test or control sequence comprises asequence further determined by extracting a context sequence containingeach locus in the representative set of loci.
 17. The method of claim16, wherein the test or control sequence comprises a sequence furtherdetermined by combining in silico the extracted context sequences. 18.The method of claim 17, wherein the test or control sequence comprises asequence further determined by attaching one or more sequencing adaptersto the combined sequence.
 19. The method of claim 12, wherein the testor control sequence comprises a sequence further determined by findingco-occurring variant locations present in at least three and no morethan twenty sequencing runs included in a training set of sequencingruns.
 20. The method of claim 12, wherein the test or control sequencecomprises a sequence further determined by finding co-occurring variantlocations present in at least five and no more than fifteen sequencingruns included in a training set of sequencing runs.